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"The computer display screen will be mankind's new home." 

- Ted Nelson, Dream Machines, The Flip Side of 
Computer Lib, Microsoft Press, 1974, 1987. 
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In this year of 1991, we are in the 21st 
Century. 

Advances in adaptation of technology are 
historically accentuated, benchmarked, and 
propelled forward, as distasteful as it is, 
by wars. A prime example is the 19th 
Century American Civil War. It is 
commonly accepted, in historical context, 
that this war was the first war of the 20th 
Century simply through its rapid 
technology developments In weapons and 
weapons support systems. The repeating 
carbine, Gatling gun, iron clads, telegraph, 
/.e., electric communications, submarines, 
balloons, l.e., lighter-than-air craft, 
railroad transportation, weapons 
metallurgy and ballistics, photography, 
i.e., imaging technologies, are examples of 
the Civil War technology adaptations that 
influenced the thrusts of warfare for most 
of the 20th Century. Peacetime growth of 
technologies was in turn propelled forward 
by the wartime developments, e.g., 
expansion of electricity-driven 
communications, image presentation, 
railroad and automotive transportation, 
distribution systems for commodities, 
manufacturing technologies, foodstuffs 
preservation, aircraft developments, and 
on and on. 

In our common experience of today, the 
Persian Gulf War of January, 1991, 


because of its unprecedented technology, is 
the first war of the 21st Century. It is the 
benchmark war that has kicked us across 
the threshold to the 21st Century. Up until 
this war, a plethora of reports, books, and 
commentaries coming out of the Federal 
government, private industry, academia, 
and the press constantly questioned the 
world position of the United States in its 
predominance in technology. The 
technology used in the Persian Gulf War, 
successfully proven under fire, confirms 
U.S. technology prowess. 

NASA had a major part in the success of 
American technology in the Persian Gulf 
War, as indicated in its summer 1991 
issue of NASA Update - State and Local 
Newsletter .* A brief article titled 
"Advanced Technology Hastens Victory in 
Gulf Conflict" (p. 3), states: 

Victory in the Persian Gulf conflict was 
a triumph of advanced technologies. 
Many of those technologies had roots in 
aeronautics and space research and 
development that NASA and the 
U.S. military establishment have 
conducted or sponsored over the years. 


* NASA Office of External Relations, Defense 
and Intergovernmental Relations Division. 
NASA Update: State and Local Newsletter 5, 
1 (1991). 
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. . . Some of the types of systems that 
contributed to the speed with which the 
war was terminated and that were 
possible only because of advanced 
technology development by NASA and the 
military are: 

• Visual and infrared imaging systems 
and the related systems for interpreting 
and presenting information collected by 
the imaging systems. Those systems 
contributed significantly to the accuracy 
achieved in reconnaissance, 
surveillance, and targeting. 

• Satellite-based communications 
systems that provide transmissions to 
and from remote locations. Essential to 
the coordinated, effective operations that 
speedily provided victory, they had the 
side effect of turning the conflict into a 
" media event." 

• Computers that are small, rugged, and 
portable. These ubiquitous marvels of 
the space age made possible any number 
of Improvements in planning and 
execution not previously possible. 

• Propulsion and guidance systems, 
which enable "smart munitions" to 
attack defended targets with devastating 
accuracy and the Patriot missile to 
become the “Scud buster." 

The " bottom line" for advanced 
technology In the U.S. arsenal is the 
speed w ith which hostilities were 
successfully concluded and the avoidance 
of unnecessary injuries and deaths of 
U.S. personnel. 

A good part of that technology involved 
machines making decisions and conveying 
information based on those decisions to 
people, vis-S-vJs people making decisions 
based on raw data provided by machines. 
This is the new environment, not only for 
warfare, but now also for peacetime. In 
the first NASA STIP Quo Vadimus document 
of May 1991 (BLAD91), the author, Mr. 
Walter Blados states: 


...we find ourselves living in a new 
environment... . New relationships and 
new opportunities are resulting from 
the increased availability of information 
and the stimulation and expanding 
opportunities it brings. There will be 
new individual and organizational 
relatio nship s, new methods of doing our 
work, and new environments in which 
we will carry out our work. 

A part of our 21st Century environment 
and its "new methods of doing our work" is 
computer-driven multimedia, an ever- 
expanding means of conveying information 
to people. 


WHAT IS MULTIMEDIA? 


Multimedia is computer integration and 
output of text, animation, audio, video, and 
graphics. 

You sit at your computer screen and watch 
and hear the action. You choose what you 
want to take place. You lis ten to what 
someone has to say as you watch him speak; 
you hear music as background to the 
multimedia presentation; with scientific 
experiments and studies you witness the 
experiment and its attendant results in all 
their forms - visual and auctorial; you 
ex am ine ch anges, shapes, and form s based 
on those changes you yourself select. 


A NASA engineer at La ngley Research 
Center re view s a wind tunne l test. Th e 
engineer inserts an optica l disc in to a 
drive, "and observes the actual test as it 
took place. This includes sound and 
visuals, in addition to the manuals needed 
to run both the wind tunnel multi me dia 
system, and the actual wind tunnel system 
itself. An em bedded advisory system not 
only stores experts' knowledge and 
experiences, but can be added to by the 
engineer. The engineer proceeds with 
visualizations of computerized model data 
from the test - pressure distribution in 
both actual test data and computer 
simulation data. The test platform is re- 
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oriented on the screen to see the effects of 
the test from differing vantage points. He 
changes test factors to see what changes 
would take place in the test results. Both 
sound and color, in addition to the video 
representation, convey data to the 
engineer. The engineer, in reviewing this 
earlier test, makes decisions on how to 
proceed with a new test. 

A NASA employee, after a long day's work, 
goes home - and to relax, turns on his 
machine. The machine has downloaded the 
day's soap operas. In fast-forwarding 
through the programs, he selects clips, and 
stores them separately on the hard disk. In 
reviewing the clips, he further selects 
portions of them, and then joins the clips 
into a media document that he is creating. 
The media document is a "paper" he is 
composing for his Doctoral thesis on The 
Effect of Human Dramatization of 
Intersocietal Problems of People as 
Portrayed by Cable Television Services. 
The thesis will exposit the creator's 
premises through contextual visual and 
auditory information; text will be 
minimal, and that probably will be spoken 
by the creator, likely as voice-overs, less 
in lecture form. The creator will also 
create interactive elements in the thesis, 
for the referee/viewers to explore and 
compare variances to the premises. 

A medical doctor assistant on a NASA space 
station is faced with a crewman accident 
emergency, a sliced off thumb, and needs to 
review texts and procedures before 
performing a needed reparative operation. 
There are several progressive multimedia 
scenarios that can be imagined, all based on 
technologies available and emerging today. 

• Scenario A - The doctor assistant 
refers to his onboard, miniaturized 
medical reference system, comprised of 
the major medical references in U.S. 
practice. This system is replete with 
page images of the original texts, with 
the original drawings and photographic 
illustrations, including their color. 
The texts are rapidly searchable 


("hypertext") simultaneously, and 
therefore easy to cross-correlate. 

There is also voice and motion picture 
instruction on medical procedures. 
After conferring with the system and 
with "medical emergency central" -- 
Kennedy? Johnson? - the doctor 
assistant proceeds with the reparative 
procedure. This may take several 
hours. 

• Scenario B - In conjunction with the 
above, operation is closely monitored 
by earth-based medical staff following 
an analysis of televised images of the 
injury condition, with an automated 
program run-through of the reparative 
procedure based on those images and 
virtual reality technology, and 
retransmitted to the doctor assistant on 
the space station. All this happens in 
less than an hour. 

• Scenario C - Again in conjunction 
with all the above, all images of the 
situation are fed into a robotic 
cyberspace technology system - the 
system in processing the images and 
medical programs, monitored by the 
space station doctor assistant and the 
earth-based medical staff, will 
conclude the needed procedures for the 
operation and perform it. Cyberspace 
technology "is a type of interaction 
simulation which includes humans as 
necessary components. A cybernetic 
simulation is a dynamic model of a 
world filled with objects that exhibit 
lessor or greater degrees of 
intelligence." [WALS91] With this 
combination of imaging-cyberspace- 
robotic technologies, the reparative 
processes take place within ten 
minutes. 

INCREDIBLE, YOU SAY? 

Multimedia is here today. All the media 

elements involved in the above scenarios 

are alive and well - "cyberspace" 

technology, for example, has been around 
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and in development since the 1960s. 
Listen to what major movers in the 
computer industry are saying -- 

"Multimedia will change the world in the 
1990s as personal computers did in the 
1980s." - John Scully, Apple 

"Multimedia will be bigger than 
everything we do today." 

- Bill Gates, Microsoft 

"We're going to put a computer in every 
home... with multimedia. " 

- James Cannavino, IBM 

Do you get the feeling that there is 
something highly evolutionary going on in 
the realm of information? I think the very 
basic idea of multimedia is that it is the 
stage in computer and Information 
technologies that will give us access to 
experience. This may be seen more 
easily within the context of the stages of 
information access : 

• 1 Access to Data Numeric, calculatory 

• 2 Access to Information Textual results 

via coordination of terms 

• 3 Access to Knowledge Reviewing 

and reformatting i _ ele ctronic text 

• 4 Access to Experience Cognitive and 

sensory processing 

Each of these access capabilities have 
expanding sets of tools underlying them to 
provide the capabilities, with each access 
capability folding into the next level to give 
greater expanded access. - = 

It is generally accepted that the seminal 
article setting down the ideas of what are 
now multimedia processes was Vannevar 
Bush's "As We May Think," appearing In 
Atlantic Monthly ,* July 1945. In this 
article Bush, in giving form to ideas 
possible at that time, suggested creating 
linkages between documents, which in turn 


Vannevar Bush. "As We May Think." 
Atlantic Monthly 176, 1 (July 1945); 101-8. 


would be brought to a screen as rapidly as 
possible according to those linkages. In his 
time, he was thinking about microfilm 
Images. For our time, read automated text, 
pictures, movement, and sound, all 
documents in our expanded concepts. 

WHAT’S HAPPENING TODAY? 

The forces of the market place are the best 
Indicators for the use and success of 
multimedia programs. The following uses 
of private sector multim edia, shown here 
as examples of commercial multimedia 
applic ations , has been compiled and 
provided by Mr. Thomas A. Wheeler of 
Multimedia Design Corporation [WHEE91]: 

• Northern Telecom is using 
multimedia to train their 20,000 
employees at about 1/1 0th the cost of 
traditional training methods. (MacWeek 
article). 

• Ameri can Airlines has a 
multimedia proj ect on the boards to 
provide initial and refresher training for 
50,000 employees. They expect "50% 
reduction in training time per employee", 
"reducing 300 hours of lessons to 152" 
hours (MacWeek, August 14, 1990). 

• GTE North, Inc., is using, 
multimedia to teach workers how to fix 
telephone cables. "The r etention rate has 
to be three or four times higher than 
lectures and pencil testing" (Business 
Week, October 9, 1989). 

• General Motors Buick Division 
has developed a multimedia pr esentation 
that allows prospective customers to 
browse an electronic brochure, on a PC, 
through descriptions and animated pictures 
of cars, compl ete with engine sounds. The 
marketpl ace r esults o f such a broch u re 
were that 12% of those who bought cars 
after receiving t he e lectronic brochure 
ended up picking Buick - this was about 
double its usual market share (Business 
Week, October 9, 1989). The electronic 
brochure includes car models, model 
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options like color, wheels and styling, and 
spreadsheet operations that calculate 
payments and make price comparisons with 
competitor cars. This brochure is 
interactive with the customer, and 
consequently the customer spends more 
time with that than with a handful of paper 
brochures. 

• Steelcase Inc. uses an electronic 
brochure to launch a new line of office 

furniture. Customers "can get to 

information more conveniently than by 
pawing through pages of [paper] 
brochures" (Business Week, October 9, 
1989). Customers and designers move 3D 
renderings about a computer screen to see 
how different furniture looks in a specific 
room, while a voice provides product 
details against a musical background. 

• The American Heart Association 
is distributing a set of digitized interactive 
lessons to medical schools on how to manage 
blood cholesterol levels. The program 
gives students an audiovisual rundown 
detailing how cholesterol moves through 
the blood and liver. 

What other commercial effects does private 
sector multimedia have in addition to those 
noted above? A study done by Duthie and 
Associates [WHEE91] has compared the 
costs of different training methods, based 
on cost per student for 1,000 students. 
Their cost figure study results are: 


Classroom 

$8,900 

Videotape 

482 

Cassette 

459 

Workbook 

409 

Videodisc 

403 

Computer-based training 

226 

Interactive Multimedia 

132 


TERMINOLOGY, OR, 

CONFUSION & go go 

"Multimedia" and "Interactive Multimedia" 
are now the catch phrases in the market 
place. There are several other terms, 


though, that are also used to designate these 
processes, or are incorporated in the 
multimedia processes. These terms are 
"Hypermedia," "Hypertext," "Intermedia," 
and "Imaging." In the interest of some 
standard acceptance of the terms, I would 
like to pick up the following explanation 
from one of the papers in the book, 
Learning with Interactive Multimedia 
[AMBR90]: 

A note about terminology: As is often the 
case in new fields, no consensus exists 
in the multimedia field about what 
various terms mean or what to call 
certain items that clearly need names. 
"Multimedia," " hypermedia ," and 
" intermedia " are often used 
interchangeably to describe the same 
thing. In this [paper], I use the term 
" multimedia " to describe any software 
using the four essential communications 
media - text, audio, images, and logic - 
- whether on a media computer or an 
analog video-computer hybrid. I use the 
term "hypermedia" to describe a subset 
of multimedia that has "hyper" 
characteristics: various modules linked 
in a nonhierarchical, nonlinear 
structure. . . . [OSBO90] 

Multimedia and Hypermedia at this point 
are nearly interchangeable; any difference 
may be particular to an application and in 
the nuance. Interactive multimedia refers 
to the viewer making selections in the 
multimedia program that causes the 
program to go off in another direction. 
Intermedia seems to be used very little any 
longer. 

"Hypertext," in the words of the term's 
originator, Theodore "Ted" Nelson, means 
"non-sequential writing. . . . Ordinary 
writing is sequential . . . But the 
structures of ideas are not sequential. 
They tie together every which way." 
[NELS87] Mr. Nelson, who coined the 
term in 1965, means here a system of 
computer-supported, nonsequential 
information processing, primarily, of 
course, of text. One selects a term or 
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phrase, and sees how the positions of the 
term relate throughout a document or set of 
documents. The underlying principle is 
"the ability to create computer-supported 
links or cross-references permitting 
rapid, easy movement" among related parts 
of information in the text or document set 
[KAYE91]. 

"Imaging" is a widely used catch phrase 
throughout the market place. Currently 
there are two basic areas involving 
imaging technologies: business imaging and 
technical imaging. Imaging in the business 
context refers primarily to automating 
business processes and activities. This 
means basically forms and corr espond ence, 
but also extensively including any images 
and graphics that support the business 
activities. The forms and correspondence 
appear on the screen "imaged" as they 
appear in paper form. When connected to 
fax capabilities, company forms and 
correspondence, if not input into a 
receptor imaging system, fax out on paper 
in the originating company's 
correspondence letterhead and format. 
Technical imaging concerns computer- 
stored images to support scientific and 
engineering activities. It is comprised of 
everything ranging from satellite- 
originated digital data images of earth and 
space, and remotely-sensed data, to 
scanned-in maps, X-ray images, magnetic 
resonance images (MRIs), microscopic 
images, you name it. 

As a consequence of the multiplicity of 
terms, in the various examples following, 
"Hypermedia" may be correlated to 
"Multimedia"; "Hypertext" and 
"Intermedia" may be considered as features 
or effects that are incorporated in 
multimedia/hypermedia processes. 

So... 

WHAT'S HAPPENING IN NASA? 

A review of materials cited in the 
NASA/RECON Information system shows 
extensive interests across the whole of 


NASA. There are a multitude of multi- 
media/hypermedia developments going on 
in the transition from traditional linear 
information retrieval to actual information 
viewing. This is happening in what has 
been termed the multidimensional 
mformatlon space [SEPEC90J. We already 
see from above how this is changing the 
way we store, retrieve, and use infor- 
mation. Hyper-branching applications are 
being experimented with throughout the 
whole of the government, academia, and 
private Industry. The following are but a 
few examples representative of what is 
taking place In NASA. Included Is an 
example of Hypertext application, with the 
view of showing such activity as a 
component of multimedia support. The 
descriptive information of the activities 
below is extensively verbatim from the 
papers cited [KAYE91]. 

The Experiment Documentation 
Information System (EDISi . EDIS is being 
developed by Houston Applied Logic, 
Houston, Texas, for the NASA Life Sciences 
Project Division at NASA Johnson Space 
Center, Houston, Texas. It is a system 
designed to pr oduc e and control the Life 
Sciences Experiment Document (E D) 
containing large amounts of text in 
combination with tables and graphs of 
mathematical and scientific data, making 
use of hypertext concepts through 
Macintosh HyperCard.. The ED defines all 
functional objectives, inflight equipment, 
consumables, measurements, ground 
support, and test sessions, along with the 
expected results of the experiments. The 
ED consists of 16 chapters plus appendices. 
There is a fixed, or boilerplate text in 
some sections that applies to any Life 
Sciences experiment and reference table 
formats concerning experiment-specific 
text and mathematical/scientific data. 
Other sections contain experiment data 
tailored for each experiment. The EDIS is 
foreseen as being the first step in the 
automation of the process required for 
defining complete packages of Life Sciences 
experiments for the Shuttle missions 
[MOOR90]. 
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Life Sciences Interactive Information 
Recall (LSIIR), This is a study in 
hypermedia applications, being done by GE 
Government Services, Houston, Texas, for 
the Life Sciences Project Division, Johnson 
Space Center. LSIIR, through interactive 
media technologies, provides online 
information aids as a "job performance 
assistance." The technologies are 
integrated into a computer desktop 
workstation environment with which 
mission or payload specialist, the 
scientist, the engineer, and support or 
administrative people are familiar. The 
LSIIR is foreseen as providing assistance in 
Life Sciences Project missions and 
activities such as development and testing, 
science monitoring, technical lab 
activities, and mission testing. The system 
uses Mac SEs for running integrated 
applications of HyperCard, MacRecorder 
Sound System, MacDraw, MacPaint, 
Canvas, and MacroMind Director. 
MacroMind Director enhances graphics 
display and animation. Clip art and 
scanned photos are part of the system's 
information base. The system serves as a 
"trainer" or simulator. It provides the 
user with different sets of information to 
change variables during an exercise, or 
make alterations to procedures and 
configurations. LSIIR has passed its proof 
of concept stage, and is envisioned as an 
online system for electronic documentation 
and information, and electronic training 
and review in all areas of the NASA Life 
Sciences Project activity (CHRIS90]. 

Knowledge Bass Biowa.e.1 (KB.B1- 

Currently under development at the NASA 
Johnson Space Center is a hypermedia 
system for browsing CLIPS knowledge 
bases. CLIPS is C Language Integrated 
Production System, an expert system shell 
used in this case to create knowledge base 
expert systems of rules that control the 
processes of the Onboard Navigation 
(ONAV) flight control position at the 
Mission Control Center (MCC). These 
expert systems will support the ascent, 
rendezvous, and deorbit/landing phases of a 
Shuttle mission. The KBB, as a component 


program of the MCC, serves to assist in the 
verification of the rule bases of the various 
expert systems, and to augment the 
training of the flight controllers. When 
complete, the KBB will verify and browse 
the CLIPS rule bases. This system, which 
in the view of its creators is a hypermedia 
system, will include the capabilities of 
automatic creation of links based on the 
CLIPS rule structure, querying the rules 
and saving the results as a collection, and 
browsing the rule bases either sequentially 
or by using the links and collections 
[POCK90]. 

The Space Station Freedom User Interface 
Language (SSF UIL) . SSF UIL is in 
development at the Space Operations and 
Information Systems Division of the 
Laboratory for Atmospheric and Space 
Physics, University of Colorado, Boulder. 
It is designed for use by the astronauts, 
ground controllers, scientific 
investigators, and hardware/software 
engineers who will test and operate the 
systems and payloads aboard the space 
station. The IUL is object-oriented, 
English-like, supplements the graphical 
user interface to systems and payloads by 
providing command line entry, and will be 
used to write test and operations 
procedures. Hypertext is used to provide 
links between users of code (statements, 
steps, procedures, etc.) and associated 
annotation and documentation, linking code 
to object information, and linking steps 
within a procedure [DAVI90]. 

Artificiall y Intellig ent Grap hi cal Ent ity 
Relation Modeler (AiGerm) . AiGerm is a 
relational database query and programming 
language frontend for Germ (Graphical 
Entity Relational Modeling) system. These 
systems are being developed by 
Microelectronics and Compute Technology 
Corporation, Software Technology Program 
(MCC/STP), Austin, Texas. There are 
three versions of AiGerm in use: Quintus 
Prolog, BIMprolog, and MCC's Logical Data 
Language (LDL). AiGerm is intended as an 
add-on component of the Germ system to be 
used for navigating very large networks of 
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information, harnessing Prolog or LDL's 
relational database query capabilities. It 
can also function as an expert system shell 
for prototyping knowledge-based systems. 
AiGerm provides an interface between the 
programming language and Germ. When a 
user starts up AiGerm, the system builds a 
knowledge base of currently loaded Germ 
folio. The knowledge base is a collection of 
node, link, and aggregate facts. The user 
queries the database and runs programs 
that select, create, delete, inspect, and 
aggregate the nodes and links appearing in 
the Germ browser. To use AiGerm, the 
user first starts up Ge rm and lo ads the 
desired hypertext network folio into the 
Germ browser. In a knowledge base, for 
example, for each hypertext entity - i.e., 
node, link, and aggregate - AiGerm asserts 
a fact (a prolog clause). AiGerm is 
currently used in MCC/STP's DESIRE 
(DESign Information REcovery) system to 
extract information on the design code for 
software systems. Research staff are 
experimenting with AiGerm in building 
IBIS (Issue Based information Systems) - 
reasoning and decision support systems for 
softwar e design and engineering. Rockwell 
I nte rnational, an MCC/STP shareholder, 
uses AiGerm in a simultaneous engineering 
project. MCC/STP states that users of 
AiGerm can navigate Germ Networks or 
develop prototypes of knowledge-based 
h yp ermedia systems [HASH90]. 

Clinical Practice Library of Medicine 
fCPLM) . The CP LM was conceived in 
1979 by a team of medical and computer 
experts from the University of Flor ida and 
Kennedy Space Center. Since its onset, the 
system h as evolved from a mainframe- 
based text database to a microcomputer- 
based hypermedia system that supports 
both text and high-resolution medical 
images. The design changes necessary to 
expand the system to include sound and 
animation are now being delineated. 

The CPLM system is currently a 
computerized, rapid-reacting, medical 
reference system that could be placed 
aboard a long-term space flight to provide 


the spacecraft physician with nearly 
instantaneous access to the most complete 
medical references on Earth. With this 
type of support system, the physician could 
be confident that he was making the right 
diagnosis. The demonstration CPLM system 
that is available now runs on an IBM PS2 
Model 80 microcomputer with a high 
resolution 851 4A Display and a 1 gigabyte 
disk drive. The system is programmed in C 
under Microsoft Windows. The system 
contains a variety of medical texts 
including the STI Program's special 
publication NA SA SP-3006, the 
"Bioastron autics Data B ook." The CPLM 
system is written to allow expansibility to 
the full capacity of the available storage 
device. Both traditional and hypermedia 
access to the information is permitted. 
T raditional Boolean search methods are 
enhanced by a parsings dictionary unique to 
each book that holds current spellings and 
root word divisions along with a lexicon 
that provides a book specific list of 
synonyms and abbreviations that 
automatically provides alternate search 
terms to the user. Word and phrase 
linkage among all documents Is provided 
initially by the University of Florida 
project team with annotations to be 
eventually added by the physician end 
users. The educational capability of the 
CPLM system may be one of its major 
benefits in addition to its ability to deliver 
complex information in a user-friendly 
fashion. 

The University of Florida development 
team leader stre sse s that the planned 
addition of voice activation, animation, and 
interactive h ar dware can make the CPLM 
system function as a fully automated 
physician's assistant. In a few years a 
miniaturized hypermedia CPLM system is 
foreseen as being built into space suits and 
carried by Earthly physicians in their 
black bags [GRAM91]. 


Virtual Interface Environment W orkstation 
fVIEWL VIEW is a NASA development with 
high visibility, that is based on Virtual 
Reality technology. Virtual reality is "the 


creation of highly interactive, computer- 
based multimedia environments in which 
the user becomes a participant with the 
computer in a 'virtually real' world" 
[HELS91]. In the words of the founder and 
first Director of the VIEW Project:* 

In the Aerospace Human Factors 
Research Division of NASA's Ames 
Research Center, an interactive Virtual 
Interface Environment Workstation 
(VIEW) has been developed as a new kind 
of media-based display and control 
environment that is closely matched to 
human sensory and cognitive 
capabilities. The VIEW system provides 
a virtual auditory and stereoscopic 
image surrounding that is responsive to 
inputs from the operator's position, 
voice and gestures. . . . this variable 
interface configuration allows an 
operator to virtually explore a 360- 
degree synthesized or remotely sensed 
environment and viscerally interact 
with the components. 

The current [VIEW] system consists of a 
wide-angle stereoscopic display unit, 
glove-like devices for multiple degree- 
of-freedom tactile input, connected 
speech recognition technology, gesture 
tracking devices, 3-D auditory display 
and speech-synthesis technology, and 
computer graphic and video image 
generation equipment. 

When combined with magnetic head and 
limb position tracking technology, the 
head-coupled display presents visual 
and auditory imagery that appears to 
completely surround the user in 3-D 
space. . . . [FISH91] 


Scott S. Fisher, Project Director, 1985-90, 
following which he jointly founded Telepresence 
Research. Mr. Fisher holds an M.S. in Media 
Technology from MIT. (Source: Virtual Reality 
[HELS91 ]) 


ACADEMIC ACTIVITIES: TWO MAJOR 
EXAMPLES 

PROJECT EMPEROR-I . This is a well 
known hypermedia project, merging 
microcomputer and videodisc hybrid 
technologies. It has been ongoing since 
1984. It is a major research and 
development project which demonstrates 
how new technologies enhance better 
understanding and appreciation of a 
subject, in this case Chinese humanities, 
by delivering a large-scale online (real- 
time) hypermedia, multi-formatted, and 
multi-dimensional information simply not 
possible in sequential-formatted systems. 
The current hypermedia system includes 
an interactive information delivery model 
for providing, at rapid speeds measured in 
fractions of of a second, requested relevant 
information in any format - visual, audio, 
textual - as selected by the viewers at 
their pace and choice, including at the point 
of need. The project now includes: 

• Two 12 inch NTSC CAV videodiscs, 
entitled "The First Emperor of China: Qin 
Shi Huang Di." 

• Interactive courseware, at both a lay 
public and a serious researcher levels. 
Prototype courses have been developed for 
Digital Equipment Corporation's I VIS 
systems and for IBM PC compatibles. Later 
systems now include the Apple Macintosh 
Mac lls. 

• Electronic image databases for IBM 
compatibles and Mac lls. Further 
development efforts have taken place with 
SOPHIATEC, Nice, France, and with the 
Project Athena of Massachusetts Institute 
of Technology involving a powerful 
multimedia image system using DEC'S 
proprietary MUSE software for high-end 
machines such as DEC'S MicroVax and IBM 
RTs. The EMPEROR-I hypermedia system 
has also been looked at for use on Sun3s and 
Sun4s. 

• High resolution imaging digitization 
and electronic imaging has been performed 
on a Sun3-160 using OASIS software. 

• Converting and creating large textual 
files with images and Chinese characters 
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using MicroTek's MSF-3000 image 
scanner and INOVATIC's Readstar II Plus 
optical character recognition software. 
Digital textual files are kept in the hard 
disks, but when the data approaches 400- 
500 megabytes, CD-ROM can be produced. 

This project, housed at Simmons College, 
Boston, Massachusetts, and aided by many 
interested resources, both industry and 
academic, is a masterful development. The 
goal of the project's director, Professor 
Ching-chi Chen, is to show that computer 
power, storage technology, and software 
are now all available, at affordable cost, to 
provide the opportunities for innovative 
experimentation of ideas in education, 
training, research and development in 
nearly every subject field [CHEN88, 
CHEN90}. 

Intermedia . Brown University's Institute 
for Research in Information and 
Scholarship (IRIS), Providence, Rhode 
Island, has developed a powerful multi- 
user hypermedia software that allows 
professors, students, and other knowledge 
workers to create and follow links between 
electronic documents for different types. 
This system is named "Intermedia." This 
project defines hypermedia as the dynamic 
linking of data such that related data is 
easily accessible although the actual pieces 
of data may be stored in different physical 
locations. In theory the data can be any 
type, such as text, graphics, spreadsheets, 
video, or audio. Intermedia provides a 
desktop environment similar to that found 
on the Macintosh. The desktop contains 
applications (or tools) such as a wo rd 
processor, a structured graphics editor, a 
historical timeline editor, a scanned-image 
view, an animation editor, a videodisc 
controller, and a viewer that displays and 
rotates three dimensional models. Users 
(now termed "viewer” or "authors"), with 
the tools just enumerated, enter data and 
link significant items of Information 
together for a contextual viewing of that 
information. 

Because of the extensive differences in the 


storage sources of the information, the 
Intermedia development incorporated two 
new concepts in the handling of the 
information, the "anchor" and the "proxy." 
The anchor concerns maintaining 
consistency across the applications; an 
anchor is a specific selection of data, a part 
of a document, with the surrounding 
information used to understand its 
significance. When a user follows a link, 
the document window opens to the size and 
location on the screen most recently saved, 
and automatically scrolls to the sec tion that 
reveals the anchor with its surrounding 
information. The proxy is an intermediary 
concept used by the viewer for selecting an 
anchor in disparate data sources, e.g., text, 
graphics, sound. The use of the data proxy 
concept allows the viewer to visualize non- 
graphical and conceptual media, to have 
simplicity in linking media, and to extend 
system applications to related data types 
[CATL88]. 


21st CENTURY FUTURES - THEY ARE 
HERE - OR IF NOT, THEN NOT MORE 
THAN 10 MINUTES AHEAD OF THE 
TROTH 

There is so much technology on the boards 
and coming up that the subject of futures is 
in itself expansive, much more than just in 
the imagination. Science fiction in today's 
terms must be more than outlandish, 
because a good part of yesterday's sci fi is 
here. Examples: the following appear in 
The Futurist, September-October 1990, 
Column: "Tomorrow in brief" -* 

• A natural appearance prost he sis 
(e.g., artificial leg) made of carbon fiber 
an d contai ned jn a soft continuous covering 
gives amputees greater mobility to the 
extent that they may take part in strenuous 
activities such as rock climbing, cycling, 
and squash. 

• Customized computers are making 
more and more inanimate objects into 
smart appliances, tools, and toys. An 
exotic example is the "skidometer", a 
microprocessor embedded in skis to 
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measure speed and elapsed time on the 
slope. It is expected that major future 
growth of microprocessor applications will 
be in household appliances. Such 
applications will cause the use of these 
devices to grow from about 300 million in 
1989 to 500 million in 1995. 

• A recent OTA report provides 
recommendations on reducing the 
tremendous costs of spacecraft. 
Alternatives suggested are: 

Fatsats - spacecraft made of heavier, 
cheaper materials. 

Lightsats - smaller spacecraft with 
fewer capabilities, thereby costing less to 
build and launch. 

Microspacecraft - small, rugged 
satellites containing very small 
instruments, which can be launched 
several at a time from cannon-like 
launchers. 

• The Paris Transit Authority - RATP 
- has begun testing a new train it hopes to 
be the "metro of the year 2000." It is not 
sectioned off into single cars, but a long, 
single train, nicknamed "The Boa," 
allowing its 890 passengers to walk from 
one end to the other. It is expected to 
replace all first generation, non- 
pneumatic trains in the Paris metro 
system between 1998 and 2010. 

• Seiko Epson Co. Ltd. is producing a 
hand-held device, about the size of a TV 
remote control, that translates English 
words into Japanese. You run it across a 
page; it looks up each word and matches the 
English words to 30,889 Japanese words. 
(Product: Epson TRAN PRO-1000.) 

• A compact, desktop-size robot arm 
could help students jot down notes and 
conduct chemistry experiments. Developed 
by Systems Control of Middlesborough, 
England, it can be operated through an 
IBM-compatible computer and uses 
standard, pretaught sequences. Students 


can teach the robot new sequences though 
the robot's keypad or the computer 
keyboard. 

An item of note, of interest to NASA people, 
concerns the development of solar 
sailcraft. In the column "Future scope" of 
the same issue the following appears: 

Solar Sailcraft Will Race to Mars - To 
honor Columbus, racers set sights on a 
" new world " - Solar sailcraft from six 
nations will compete in a race to Mars in 
1992 to commemorate the 500th 
anniversary of Christopher Columbus's 
discovery of the New World. ... The 
sails, which fly on the flow of photons 
from the sun, will be of various shapes 
and sizes ... . Most sails will be made of 
material that is thinner and lighter than 
plastic food wrap. Folded into small 
canisters, the sails will be launched by 
rocket into high Earth orbit, where they 
will unfurl. Source: U.S International 

Space Year Association... . 

Let's narrow the field to the mystique of 
information and its conveyance through 
multimedia. 

Electronic City Hall - You will register to 
vote, sign up for racketball, and lodge a 
complaint with the mayor, at an electronic 
kiosk. Currently, in Mercer Island, 
Washington, an electronic information 
booth, called "The 24-Hour City Hall," has 
been set up near a grocery store to give 
citizens 24-hour access to their municipal 
government. Developed by Public 
Technology, Inc. (PTI), it uses an IBM 
microcomputer and multimedia technology 
to give information on town meetings and 
local government services. More 
electronic city halls are being set up in 
other U. S. communities. ( The Futurist, 
September-October 1990, p.6) 

Computer with "Brain Waves" - an IBM 
scientist and colleagues at Columbia 
University designed a supercomputer 
model for studying the area of the brain 
called the hippocampus, an area associated 



with epilepsy. The model unexpectedly 
began producing its own electrical waves, 
surprisingly similar to the brain waves of 
the hippocampus. If this is an accurate 
simulation, "experiments can be done on 
the model as if it were an organism in its 
own right." { The Futurist, September- 
October 1990, p.6) 

From the magazine Verbum ,* Winter 
90/91 : 

• Mind Ware - M C £ ■ and MindsEve 
Svnerqizer (for the IBM). Light & Sound 
Research, Pasadena, CA and Synectic 

Systems, Seattle, WA. These are f W0 

brain-wave controllers, with a 
combination of special glasses and 
earphones that deliver strobe lights and 
sounds to entrain users' brain rhythms. 
The machines are based on the same 
rese arch that associated different brain 
waves with various states of consciousness. 
Each can induce either the alpha state for 
quick relaxation or a body-relaxed but 
brain-alert theta state for enhanced 
learning via cassette tape. MC2 by Light & 
Sound Research provides a microchip with 
ten preset light/sound combinations built 
into a controller box that its goggles and 
earphones hook into. It does not link with a 
personal computer system and is not user 
programmable. Synectic Systems' 
MindsEve Svneraizer provides a PC board 
that fits into a slot on an IBM 
PC/AT/XT/386 or compatible, along with 
goggles, headphones and software. The 
specific light/sound combinations are 
programmable by the user. In fact, each 
eye and ear may be programmed 
independently. A separate kit lets 


* Verbum 4.3, Journal of Personal Computer 
Aesthetics, Winter 90/91. "This is the 
magazine that helped spawn a revolution in the 
world of personal computers by proposing that 
the creative spirit could work its magic through 
silicon, that the human hand and mind could use 
digital media in the making of True Art." (Issue 
cover) 


developers program applications in the C 
language. (Verbum, p.44) 

• Video - DigiVideo (for the Mac). Aapps 

Corp., Sunnyvale, CA. This color real- 
time video board allow the user to display 
and capture 24-bit images for actual 
computer processing, whether in 

teleconferencing or multimedia 

applications. The board includes ah 
improved tuner and supports 98 channels. 
It can also display in black and white mode 
using 128 gray levels. Also contained in 
the package Is Prefaced Publisher's 
Exposure, a color Image grabber and color 
paint CDEV. (Verbum, p.47) 

• HyperMedia - ADDmotion (for the Mac). 
Motion Works, Vancouver, BC, Canada. 
ADDmotion adds modules to HyperCard 2.0 
that allow usersZlP create animated 
sequences. The software lets yob edit and 
create Julf color pictures, animate them 
and add four-track sound. Object-oriented 
and interactive anima tion allow flexible 
animation production. Cues in ADDmotion 
are passed back to HypsiOlini, providing 
complete interaction with the HypeilMk 
programming language. (Verbum, p.42) 

• Verbumalia : Cool Stuff To Order - 
Verbum Interactive . The first edition of 
the Verbum CD-ROM. Requires Macintosh 
II with 5 MB RAM and CD-ROM drive. 
Incredible interactive magazine in color 
and sound with a Oallery of animation and 
multimedia works . . . software demos, 
interactive articles, [ and rnuch morel 
$49.95 (Verbum, p.60) the demo 
programs are in part advertisers in the 
magazine. A two-disc set in interactive 
format. The music on the discs will also 
play on a regular audio CD player. 

In addition to some "brave new world" 
technologies I have tried to give a small 
panoply of what's going on in the new world 
of multimedia - and have barely scratched 
the su rface. Not touc hed here are 
multimedia technologies such as 
Microsoft's Windows and its extensions, 
enduser-level authoring systems from 
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MacroMind, Authorware, and other 
companies, and IBM's magnificent 
educational interactive multimedia 
projects "Ulysses" and "Columbus." 


AN UNDERSTANDING OF THE ROLE OF 
MULTIMEDIA IN SCIENTIFIC AND 
TECHNICAL INFORMATION 

Multimedia is not an end in itself. Its role 
is apparent in Grand Challenges: High 
Performance Computing and 
Communications, a report issued in 1990 
by a committee supported by the National 
Science Foundation [CPME90]. According 
to the committee, challenges facing the 
United States concern high performance 
computing systems, advanced software 
technology and algorithms, the creation of a 
National Research and Education Network 
(NREN), and basic research and human 
resources. These are the four major 
components of the Grand Challenges 
program. They in turn break down further 
into more specific areas to be researched 
and developed. 

Multimedia is a result of and will be 
effected by the outcome of research and 
development in all four of the program 
components. The very basic purposes of 
the high performance computing program 
will be creation of advanced technologies 
for the collection and storing of 
information, the accessing and 
transmission of the information, the 
availability of computer resources (both 
local and remote) to process and re- 
process the information, and the collection 
and storage of tailored information locally 
to fill the needs of the local user. The 
information will be constituted in all its 
possible varieties -- text, numeric, 
graphic, sensory. 

Multimedia means no longer asking for 
paper or file output of text and numeric 
data. Multimedia means building an 
information base that suits its user. The 
. base will be comprised of scanned 
information. If it includes text, the text 


will be images of the print material, and 
will be textually searchable, i.e., 
hypertext searchable. But the base will be 
comprised basically of images relevant to 
the user’s information need. The images 
will be both static and motion, via 
animation and video. The base is further 
supported by sound -- vocal, noise, and 
musical; and graphics programs will 
present, process, and enhance the 
information; graphics in itself is an 
expansive and distinct universe of 
computer processing. 

It is thus foreseen that a user's primary 
source of working information will be the 
multimedia system information base that 
the user creates to serve his needs. This 
will be a local system; information and 
computing resources may be accessed 
remotely, but such activity will be 
subsidiary in a supporting capacity to the 
local multimedia base. 

The NASA Scientific and Technical 
Information Program (STIP) sees the need 
for incorporating new technologies into the 
handling of STI. As stated by the Program's 
Director in the STIP strategic plan 
(STIP91), "As we approach the year 
2000, there are many exciting 
opportunities for revolutionizing the way 
STI is managed, thereby increasing its 
value to NASA, the U.S., and our 
international partners." 

In summary, I would like to paraphrase an 
earlier paper [KUHN89]: The NASA 

Scientific and Technical Information 
Program is at the threshold of its entry 
into multimedia. Because the business of 
STIP is information, there is a vast 
opportunity for looking at multimedia 
applications, assessing their relevance to 
the handling and processing of scientific 
and technical information in the NASA 
research and development community, and 
developing multimedia implementations for 
doing so. 

As NASA STIP explores this area more 
intensively, we find an increasing critical 



requirement to coordinate and interface all 
STIP multimedia development to assist the 
NASA user in the quest for information. 
Multimedia in all its forms is establishing 
the contextual handling of user-selected, 
user-relevant information. Contextual 
handling of information, regardless of its 
source and form, and approximating the 
human manner of handling information, is 
the reason for implementing multimedia 
systems. 

In doing this paper on the' subject of 
multimedia, though, note an irony - it's 
paper, it's without graphics, it's teletype- 
style sequential. Yet this is the verge of 
the 21st Century. All the capabilities of 
multimedia expressed in this paper are 
present today. 

All we need to do is to make the jump, and 
today that's hardly a large one. 

§§§ 
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CAVEAT 

The mentioning of companies and products 


in this paper is by no means an 
endorsement. The companies and products 
noted simply serve to illustrate 
pragmatically the many multimedia 
programs and systems in use in private 
industry. 
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